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Background / Context: 


Secondary students must be exposed to technology in blended learning environments across 
general and special education settings to ensure they learn technology skills as well as continue 
to learn fundamental reading skills in order to be prepared for college and careers. Reading skills 
affect later life outcomes, such as employment status (Kutner et al., 2007) and dropping out of 
school (McGee, Prior, Williams, Smart, & Sanson 2002). Digital or blended learning is 
increasingly emphasized, as evidenced by the recent prioritization in the Every Student Succeeds 
Act (ESSA) of 2015, which states digital learning is “any instructional practice that effectively 
uses technology to strengthen a student’s learning experience’’, and blended learning is “a formal 
education program that leverages both technology and face-to-face instructional approaches”. 


As such, the integration of reading instruction into college and career readiness content, and 
delivery over digital or blended learning contexts is critical. One example of an emerging 
evidence-based online curriculum is EnvisionIT developed by the Ohio State University 
Nisonger Center (2012-2017). Table 1 shows explicit examples of how the EnvsionIT 
curriculum maps onto the ESSA definitions of digital and blended learning. 


Purpose / Objective / Research Question: 

The purpose of this study was to examine the effect of EnvisionIT on secondary student reading 
skills as measured by the AIMS Web Maze Test for 8th Grade (Shinn & Shinn, 2002). Three 
research questions guided this study: (a) What is the effect of EnvisionIT on AIMSweb8 scores? 
(b) Does this effect differ by grade? (c) Does this effect differ by length of class (semester or 
year)? 

Setting: 

The curriculum was implemented in secondary school settings including special and general 
education courses (e.g., English Language Arts, Career and/or Vocational Education, and 
Postsecondary Planning) and resource rooms in 10 schools located in Ohio and Connecticut. 


Population / Participants / Subjects: 


The total sample included secondary students with and without disabilities (n = 338). See Table 2 
for sample characteristics, stratified by intervention status. 
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Intervention / Program / Practice: 


EnvisionIT is a 12-unit curriculum centered on career readiness, information technology literacy, 
and reading (Izzo et al., 2010), and delivered through an online learning management system 
called Schoology. The lessons are teacher-directed and meant to be implemented in a blended 
learning environment. Another feature is the alignment to Common Core State Standards in 
English/Language Arts, in particular the reading for informational text, writing, and speaking 
and listening standards. The units can be taught over the course of one semester or one school 
year, depending on frequency (e.g., once per week, etc.). 


Research Design: 


A quasi-experimental pretest-posttest design with students nested within teachers was utilized to 
examine the relationship between receiving the curriculum and reading outcomes. Prior to 
implementing, intervention teachers participated in a one-day training on the curriculum. 
Comparison group teachers did not receive any training, did not use EnvisionIT, and carried out 
business-as-usual instruction that had been designated by the school, district, and state. 


Data Collection and Analysis: 


To collect pretest and posttest data, trained members of the research team visited intervention 
and comparison classrooms. The AIMSweb8 test was group-administered as a paper-based 
measure. 


A difference score, Aaims8, representing change from pretest to posttest responses was 
calculated and utilized as the dependent variable. The intraclass correlation coefficient (ICC; p) 
was calculated using parameter estimates from a random effects analysis of variance (RE- 
ANOVA) model, in order to determine whether a multilevel linear modeling (MLM) was 
appropriate. 


The first research question was answered via an intercept-as-outcome model with an intervention 
indicator. The second and third research questions were answered by entering student 
characteristic variables (e.g., grade level and intervention length). 


Using the resulting parameter estimates (e.g., t-values) effect sizes, in the form of partial 
correlation (pr) coefficients (Rosenthal & Rubin, 2003), were calculated; where 0.51, 0.36, and 
0.14 correspond to large, medium, and small effect sizes, respectively. 


All analyses were estimated in Mplus, version 7.3 (Muthén & Muthén, 2015) using the full 
information maximum likelihood estimator. 


Findings / Results: 


An ICC of 0.129 was calculated for Aaims8 and was found to be distributed Aadims8 ~ N(yoo = 
0.516, Too = 5.69). The inclusion of the intervention status indicator, represented by yo; resulted 
in a pseudo 1° of 0.32; due to the RE-ANOVA 1 estimate being reduced to 3.86. The expected 
change from pretest to posttest for the comparison group was found not to be significantly 
different from 0 (yoo = -1.262, SE: 1.106, p = 0.254); whereas, those who received the curriculum 
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experience a significant boost in their change score (yo1 = 2.818, SE: 1.375, p < 0.05); therefore, 
the model implied change in AIMSweb8 from pretest to posttest is calculated via the linear 
combination of yoo and you, 1.56. 


After the addition of the within level predictors (e.g., grade and class length), a large effect was 
detected for the intervention status indicator. Thus, those in the reference groups (e.g., 9" 
graders) experienced a 3.11 boost in AIMSweb8 score (yo: = 3.11, SE: 1.31, p < 0.05; pr = 0.55); 
whereas, those in the 11" grade receiving the intervention and falling into all other reference 
categories, experience a boost of 3.88, corresponding to a small effect size (y4o = 3.88, SE: 1.64, 
p < 0.05; pr = 0.14); these students experience, on average, a boost of 6.99 points. See Table 3 
for parameter estimates from this final model. 


Conclusions: 


These results are promising; however, several limitations exist. Primarily, the study was not 
randomized; therefore, threats to internal validity exist. In this vein, the settings in which the 
curriculum was disseminated and the dosage was not controlled for; rather, class length was 
found to be non-significant (y19 = -0.99, SE: 1.96, p > 0.05); and fidelity of implementation was 
not investigated. Despite these limitations, EnvisionIT shows promise as a viable intervention 
that emphasizes college and career readiness and literacy skills delivered in a blended learning 
environment. Future research studies should prioritize these areas in order to more rigorously test 
the efficacy of the curriculum. 
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Table 1 


Alignment of Every Student Succeeds Act (ESSA) to EnvisionIT Curriculum 


Every Student Succeeds Act (ESSA) 
21 U.S.C. 812(c) 


EnvisionIT Tools and Practices 


“*(3) DIGITAL LEARNING.—Refers to any instructional practice that effectively uses technology to 
strengthen a student’s learning experience and encompasses a wide spectrum of tools and practices, 


including — 


“*(A) interactive learning resources, digital 


learning content [...], software, or 


simulations, that engage students in academic 


content; 


“*(B) access to online databases and other 
primary source documents; 

“*(C) the use of data and information to 
personalize learning and provide targeted 
supplementary instruction; 


“*(D) online and computer-based 
assessments; 


“*(E) learning environments that allow for 
rich collaboration and communication; 


“*(F) hybrid or blended learning, which 
occurs under direct instructor supervision 
[...] through online delivery of instruction 
with some element of student control [...]; 
and 

“*(G) access to online course opportunities 
for students in rural or remote areas. 


Delivers content via the Schoology LMS or Google 
Drive that teaches information and 
communications technology literacy skills to 
engage students in career research using credible 
Web sources and databases 

Teaches students how to navigate educational, career 
and college websites 

Facilitates age-appropriate transition assessments to 
personalize learning so students explore college 
and career options aligned with their interest, 
personality and learning styles 

Students complete online age-appropriate transition 
assessments, such as the VARK Learning 
Questionnaire, O*NET Interest Profiler, and unit 
quizzes 

Students participate in group discussions, adult support 
and peer review activities, and blogs to share 
results of age-appropriate transition assessments 
and career research 

Students work independently to read content and 
complete activities and assignments, resulting in a 
comprehensive Transition Portfolio 


Implemented in rural school districts with Internet 
access; students can access digital curricula at 
school, home, library — wherever student can 
access the Internet 


“SEC. 4102. DEFINITIONS. 


““(1) BLENDED LEARNING.—Refers to a formal education program that leverages both technology- 
based and face-to-face instructional approaches — 


“*(A) that include an element of online or 


digital learning, combined with supervised 


learning time, and student-led learning, in 


which the elements are connected to provide 


an integrated learning experience; and 


“*(B) in which students are provided some 
control over time, path, pace. 


Teaches students to navigate career based websites 
based on age-appropriate transition assessments 


Students work independently on activities after content 
is delivered by either a teacher or independently by 
students 


Students work independently to read content and 
complete activities and assignments, resulting in a 
comprehensive Transition Portfolio 
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Table 2. Sample Characteristics 


Intervention | Comparison 
Number of Teachers 11 7 
Number of Students 223 115 
No Documented Disability 40% 48% 
On IEP 57% 48% 
On 504 plan 3% 4% 
Disability Categories 
Learning Disability 27% 24% 
Autism Spectrum Disorder 10% 4% 
Attention Deficit/Hyperactivity Disorder 8% 2% 
Chronic Health Condition 8% 12% 
Psychological Psychiatric Disorder 4% 4% 
Intellectual Disability <1% 2% 
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Table 3 


Fixed Effects Estimates (Top) and Variance-Covariance 
Estimates (Bottom) for Final Model of the Effectiveness of 


EnvisionIT on aimsweb8 


Parameter Final Model Tequivalent 
Fixed Effects pr 

Intercept -3.40 (1.43) 0.55 
Level | 

(Student Specific) 

Year Long -0.99 (1.96) 0.03 

Lunch 0.88 (0.83) 0.06 

10th Grade 1.45 (1.76) 0.05 

11th Grade 3.88* (1.64) 0.14 

12th Grade 3.11 (2.19) 0.08 
Level 2 

(Teacher) 

EnvisionIT a1 (131) 0.55 

Random Parameters 

Level 2 

Intercept (100) 2.86 (1.86) 0.39 
Level | 

Intercept (6°) 39.13 (3.36) 0.57 

-2*log likelihood 1852.2 
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